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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1)^ Responsive to communication(s) filed on 20 October 2003 . 
2a)D This action is FINAL. 2b)E3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 1-47 is/are pending in the application. 

4a) Of the above claim(s) 44-47 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) IE Claim(s) 1-10. 13-22.28.29 and 33-37 is/are rejected. 

7) KI Claim(s) 11-12. 23-27. 30-32. and 38-43 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 13 June 2003 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (0- 

a)DAII b)Q Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.Q Certified copies of the priority documents have been received in Application No. . 

3.0 Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)), 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1.78. 
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DETAILED ACTION 

1. The amendment received on October 20, 2003 has been entered in full. An updated 
search led to the discovery of pertinent prior art. Therefore, this action is non-final. 

Response to Arguments 

2. Applicant's arguments filed October 20, 2003 have been fully considered. A summary of 
these arguments is provided below. 

37 CFR 1.75 Claim Objections 

Summary of Argument: The applicant argues that the objections to the claims should be 
withdrawn in light of the amendment to claim 24 to change its dependency from claim 21 to 
claim 23 (see page 9, section 2 of applicant's remarks filed October 20, 2003). 

Examiner's Response: The examiner agrees. The objection to claims 24 and 25 has been 
withdrawn. 
Prior Art Rejections 
35 U.S.C. 102(b) rejections 

Summary of Argument: The applicant argues that Birchfield (the article titled "Elliptical 
Head Tracking Using Intensity Gradients and Color Histograms") discloses using a fixed model 
instead of automatically learning a color-based object model as required by the independent 
claims (see pages 10-15, section 3.1 of applicant's remarks filed October 20, 2003). 

Examiner's response: Applicant's arguments with respect to the 35 U.S.C. 102(b) 
rejection have been considered but are moot in view of the new ground(s) of rejection. 
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35 U.S.C. 103(a) rejections 

Summary of Argument: The applicant argues that the combination of Birchfield and 
Koller et al. (the article titled "Using Learning for Approximation in Stochastic Processes") fails 
to disclose the applicant's claimed element of automatically learning color-based object models 
and is therefore improper (see page 16, section 4.1 of applicant's remarks filed October 20, 
2003), 

Examiner 's response: Applicant's arguments with respect to the 35 U.S.C. 103(a) 
rejection have been considered but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC §102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

4. Claims 1-7, 15, 16, 19, 28, 29, 33-35 rejected under 35 U.S.C. 102(b) as being anticipated 
by Wren et al. (the article titled "Pfinder: Real-time Tracking of the Human Body"). 

As applied to claim 1, which is representative of claims 15 and 16, Wren et al. discloses a 
system for tracking at least one object in at least one sequential image, comprising: a general 
purpose computing device (see page 6, section 8: The reference describes the use of a 200 MHz 
R4400 processor Indy with Vino Video (i.e. a general purpose computing device).); and a 
computer program comprising program modules executable by the computing device, wherein 
the computing device is directed by the program modules of the computer program to: (a) 
generate a state estimate defining probabilistic configurations of each object for each sequential 
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image (see page 2, section 3.1: The reference describes determining a the blob spatial 
characteristics of an object in an image. This corresponds to generating a state estimate defining 
probabilistic configurations (see equation (1)).); (b) generate observations of pixel color for each 
sequential image (see page 2, section 3.1: The reference describes that each blob has a color 
component (i.e. generating observations of pixel color).); (c) automatically learn a color-based 
object model using the state estimate and the observations (see page 2, section 3.1 : The reference 
describes that the statistics of the blob are recursively updated to combine information contained 
in the most recent image with knowledge contained in the priors (i.e. automatically learn a color- 
based object model).); and (d) automatically track each object using the learned color-based 
model with a color-based tracking function (see page 3, section 3.3: The reference describes the 
use of an analysis loop to track an object in an image based on the person model (i.e. color-based 
model) using tracking functions described by the equations (4), (5), and (6).). 

As applied to claim 2, Wren et al. discloses that generating the state estimate comprises 
determining the probabilistic configurations of each object using an initial image processing 
program module (see section 8: The reference describes the use of a 200 MHz R4400 processor 
Indy with Vino Video. This computer has all of the program modules including an initial image- 
processing unit). 

As applied to claim 3, Wren et al. discloses that the initial image processing program 
module employs a tracking system comprising a tracking function in combination with an object 
model for probabilistically detecting object configuration information (see page 3, section 3.3: 
The reference describes that the tracking system uses a likelihood function (equation (5)) (i.e. a 



Application/Control Number: 09/592,750 Page 5 

Art Unit: 2621 

tracking function) in combination with the blob models (i.e. object models) for detecting object 
configuration information.). 

As applied to claim 4, Wren et al. discloses that the initial image processing program 
module employs a contour-based tracking function in combination with a contour-based object 
model for probabilistically detecting object configuration information (see page 3, section 3.3: 
The reference describes the use of connectivity constraints (i.e. contour-based tracking 
function).). 

As applied to claim 5, which is representative of claim 19, Wren et al. discloses that 
generating the observations of pixel color comprises collecting pixel color information over the 
entirety of each image (see page 3, section 3.2: The reference describes that a model of the scene 
is generated that is partially based on the color distribution of each pixel in the image.). 

As applied to claim 6, Wren et al. discloses that generating the observations of pixel color 
comprises collecting pixel color information over specific portions of each image (see page 2, 
section 3.1: The reference describes that each blob (i.e. specific portions of each image) has a 
color component.). 

As applied to claim 7, Wren et al. discloses generating the observations of pixel color 
employs the state estimate to identify specific relevant regions of each image over which pixel 
color information will be collected (see page 2, section 3.1: The reference describes that each 
blob has a spatial characteristic (i.e. state estimate) which identifies a relevant region in the 
image. A color component is determined for each spatial characteristic). 

As applied to claim 28, Wren et al. discloses that generating the state estimate comprises 
processing each image with an initial object model and an initial tracking function (see page 2, 
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section 3: The reference describes a "steady- state" case in which the system has built 
representations of a person (i.e. an initial object model and tracking function) and uses them to 
initialize the system (i.e. generate the state estimate).). 

As applied to claim 29, Wren et al. discloses that the initial object model is iteratively 
replaced with the learned color-based object model and the initial tracking function is replaced 
with a color-based tracking function to improve the accuracy of the learned color-based object 
model (see page 3, section 3.3: The reference describes that the person models are continuously 
updated (i.e. iteratively replaced) to improve the accuracy of the learned color-based object 
model.). 

As applied to claim 33, Wren et al. discloses a process for gathering the sequential 
images (see page 6, section 8: The reference describes the use of a single CCD, color camera for 
gathering the sequential images.). 

As applied to claims 34 and 35, which merely disclose a computer-readable memory 
comprising a computer program comprising program modules that perform the processes of 
claims 15 and 28, Wren et al. discloses such a computer-readable memory since a computer 
performs all of the processing in Wren et al. 

Claim Rejections - 35 USC §103 



5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



I 
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6. Claims 8, 10, 13, 14, 17, 18, and 20-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Wren et al. (the article titled "Pfinder: Real-Time Tracking 
of the Human Body") and Birchfield (the article titled "Elliptical Head Tracking Using Intensity 
Gradients and Color Histograms"). The arguments as to the relevance of Wren et al. in the 
rejection of claims 1 and 15 above are incorporated herein. 

Claims 8, 10, 13, 14, 17, 18, and 20-22 call for the use of histogram processing when 
generating the color observations. Wren et al. describes that color observations are taken into 
account when determining the object model; however, the reference does not disclose the use of 
histogram processing. Birchfield, in the same field of endeavor of object tracking and the same 
problem solving area of modeling objects using color, describes the use of such histogram 
processing as detailed below with reference to each of the claims. 

As applied to claim 8, which is representative of claim 21, Birchfield discloses that 
generating the observations of pixel color comprises automatically generating a first probability 
distribution function modeled using a first histogram to represent a range of observed pixel 
colors (see section 4: The reference describes the use of a histogram for generating color 
observations of the object. A histogram, as described in the claim, is a probability distribution 
function.). 

As applied to claim 10, which is representative of claim 22, Birchfield discloses program 
module for automatically learning the color-based object model automatically computes a second 
probability distribution function modeled using a second histogram to represent a background for 
each image (see section 4: The reference describes forming an image histogram, which is a 
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histogram of the entire image minus the model (i.e. a background image). Also, as described 
above, a histogram is a probability distribution function.). 

As applied to claim 13, Birchfield discloses that automatically learning the color-based 
object model comprises performing a bin-by-bin comparison between the first histogram and the 
second histogram (see section 4: The equation for determining <X> C performs a bin-by-bin 
comparison between the model histogram (i.e. the first histogram) and the image histogram (i.e. 
the second histogram).). 

As applied to claim 14, Birchfield discloses that bins in the first histogram having values 
exceeding corresponding bins in the second histogram correspond to those color ranges 
representing the learned color-based object model (see section 4: The reference describes that the 
equation for determining <D C is used, in part, to represent the color-based object model.). 

As applied to claim 17, Birchfield discloses that a confidence measure is associated with 
the observations of pixel color (see section 4: The reference describes that an offline, as well as 
an online, histogram is obtained. The offline histogram is used as a confidence measure that the 
online histogram (i.e. observations of pixel color) is actually a histogram representation of the 
head.). 

As applied to claim 18, Birchfield discloses that the observations of pixel color are 
weighted in proportion to the confidence measure (see section 4: The equation O c is used to 
weight the online histogram (i.e. observations of pixel color) by the values of the offline 
histogram (i.e. the confidence measure).). 

As applied to claim 20, Birchfield discloses that the observations of pixel color are 
collected over specific portions of each image wherein the state estimate has a probability greater 
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than a minimum threshold level (see section 4: The reference describes determining a model 
histogram by counting the pixels inside the ellipse (i.e. collecting pixel color information over 
specific portions of each image). The ellipse corresponds to the state estimate. Therefore, an 
ellipse will only be formed if the state estimate exists, so the state estimate has to have a 
probability greater than zero (i.e. a minimum threshold level).). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Wren et al. by adding the use of histogram processing as 
described in Birchfield because the use of such processing steps allows for the color measure to 
be "more satisfied with a region containing both facial and hair color than a region containing all 
facial color" (see Birchfield: section 4). Therefore, the tracking system would provide more 
accurate results since color observations are made for more then just the facial color. 
7. Claims 9, 36, and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Wren et al. (the article titled "Pfinder: Real-Time Tracking of the Human Body") 
and Birchfield (the article titled "Elliptical Head Tracking Using Intensity Gradients and Color 
Histograms") and further in combination with Roller et al. (the article titled "Using learning for 
approximation in stochastic processes"). The arguments as to the relevance of Wren et al. in the 
rejection of claims 1 and 34 above are incorporated herein. 

Claims 36 and 37 call for the representation of pixel color information and background 
image information to be determined using a probability distribution function. This element is 
absent from Wren et al. but is described in Birchfield (see section 4: The reference describes 
using a histogram (i.e. a probability distribution function) to represent the model (i.e. pixel color 
information) and the image (i.e. background information). 
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It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Wren et al. by adding the use of probability distribution functions 
to represent color as taught in Birchfield because the use of such processing steps allows for the 
color measure to be "more satisfied with a region containing both facial and hair color than a 
region containing all facial color" (see Birchfield: section 4). Therefore, the tracking system 
would provide more accurate results since color observations are made for more then just the 
facial color. 

Claim 9, as well as claims 36 and 37, calls for the use of a Dirichlet function. The 
combination of Wren et al. and Birchfield does not teach the use of a Dirichlet function; 
however, Koller et al., in the same field of endeavor of image processing, and the same problem 
solving area of object tracking, describes the use of such a function (see column 10: The 
reference describes the use of a Dirichlet prior for use with a multinomial distribution.). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify the combination of Wren et al. and Birchfield by adding the use of 
a Dirichlet function as taught by Koller et al. because the use of a Dirichlet function "serves to 
'spread out 5 some of the probability mass over unobserved states, increasing the amount of 
exploration done for unfamiliar regions of the space" (see Koller et al.: column 11). 

Allowable Subject Matter 
8. Claims 11, 12, 23-27, 30-32, and 38-43 are objected to as being dependent upon a 
rejected base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 



Application/Control Number: 09/592,750 Page 1 1 

Art Unit: 2621 

Conclusion 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J. Miller whose telephone number is (703) 306-4142. The 
examiner can normally be reached on M-F 8:00-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo H. Boudreau can be reached on (703) 305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-4750. 

Ryan J. Miller 
Examiner 
Art Unit 2621 

Ryan J. Miller 
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